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Abstract. This paper discusses a generalization of spectral representa¬ 
tions related to convex one-homogeneous regularization functionals, e.g. 
total variation or £^-norms. Those functionals serve as a substitute for a 
Hilbert space structure (and the related norm) in classical linear spectral 
transforms, e.g. Fourier and wavelet analysis. We discuss three meaning¬ 
ful definitions of spectral representations by scale space and variational 
methods and prove that (nonlinear) eigenfunctions of the regularization 
functionals are indeed atoms in the spectral representation. Moreover, 
we verify further useful properties related to orthogonality of the decom¬ 
position and the Parseval identity. 

The spectral transform is motivated by total variation and further devel¬ 
oped to higher order variants. Moreover, we show that the approach can 
recover Fourier analysis as a special case using an appropriate ^^-type 
functional and discuss a coupled sparsity example. 

Keywords: Nonlinear spectral decomposition, nonlinear eigenfunctions, 
total variation, convex regularization 


1 Introduction 

Eigenfunction analysis has been used extensively to solve numerous signal pro¬ 
cessing, computer vision and machine-learning problems such as segmentation 
clustering m and subspace clustering [H], dimensionality reduction [5], 
and more. Recently, several attempts were made to generalize some of the prop¬ 
erties of the linear setting to a nonlinear setting. Specifically, singular-value 
analysis of convex functionals (interpreted as “ground states”) [S] and a spectral 
representation related to the total variation (TV) functional [16] were proposed. 
We examine solutions of the following nonlinear eigenvalue problem; 

Xu € dJ{u), (1) 

where J{u) is a convex functional and dJ{u) is its subgradient (precise definitions 
are given in Section [2]). We refer to functions u admitting o as eigenfunctions 
with corresponding eigenvalue A. 

In |16) a generalization of eigenfunction analysis to the total-variation case 
was proposed in the following way. Let u{t]x) be the TV-flow solution [I] or 
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the gradient descent of the total variation energy Jtv (u) , with initial condition 
fix): 

dtu = -p, pGdJrviu), u{t = 0 ) = f{x), ( 2 ) 

where 

Jrviu) = sup / udwpdx^ (3) 

IIvl|j:,°°(n)<i d 

with ip € . The TV spectral transform is defined by 

(j){t;x) :=tdttu{t;x), (4) 


where dttu is the second time derivative of the solution u(t; x) of the TV flow 
©■ For f{x) admitting ([1]), with a corresponding eigenvalue A, one obtains 
(j){t]x) = 5{t — 1/X)f{x), where 5 denotes a Dirac delta distribution. When / is 
composed of separable eigenfunctions with eigenvalues \i one obtains through 
(l>{t;x) a decomposition of the image into its eigenfunctions at t = l/Xi- In 
the general case, (j) yields a continuum multiscale representation of the image, 
generalizing structure-texture decomposition methods like [mils]. One can 
reconstruct the original image by: 

nOO 

f{x)= / (j){t;x)dt +f, (5) 

Jo 

where f = Jq f{x)dx. Given a transfer function H{t) € IR, image filtering can 
be performed by: 

^OO 

fHix):= / H{t)(j){t;x)dt + f. ( 6 ) 

Jo 

The spectrum S{t) corresponds to the amplitude of each scale: 


S{t):=\\(l){t]x)\\Li(Q)^l \cl){t-,x)\dx. (7) 

Jn 

In Fig.[T]an example of spectral TV processing is shown. For example, an (ideal) 
high-pass filter is defined by H{t) = I for a range t S [0, G] and 0 otherwise, the 
filter response is calculated using Eq. Similarly for band-pass and low-pass 
filters in different time domains (see precise definitions in [16)1. 

The analysis of eigenfunctions related to non-quadratic smoothing convex 
functionals has mainly focused on the TV functional. An extensive study was 
conducted in the past decade regarding the TV-flow, its properties like finite 
extinction time and some analytic solutions in the case of eigenfunctions, see 
|1121612411 ni4Tl4] . In |19I7] eigenfunctions related to the total-generalized-variation 
(TGV) functional [9] are analyzed. 


2 Spectral Representations 

In the following we generalize the total variation spectral approach in [TKIT^ 
in two ways. First of all we now consider arbitrary one-homogeneous convex 
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Fig. 1. Total-variation spectral filtering example. The input image (top left) is de¬ 
composed into its 4>{t) components, the corresponding spectrum S{t) is on the top 
right. Integration of the <()’s over the t domains 1, 2 and 3 (top right) yields high-pass, 
band-pass and low-pass Liters, respectively. The band-stop filter (bottom middle) is 
the complement integration domain of region 2. 


functionals J : X ^ M+U{oo} defined on Banach space X embedded into L^(I7). 
We assume that the properties of J are such that all differential equations and 
variational problems considered in the following are well-posed, which can be 
verified for standard examples such as total variation. Secondly we also consider 
alternative definitions of the spectral representation than the TV flow used in 
|15IH)] . In particular we discuss the representations introduced by a standard 
variational approach and by the Bregman iterations respectively inverse scale 
space methods (cf. |22I12I11] ). The latter has been investigated in the case of 
total variation in m- Finally we shall also discuss the issue of the spectral 
response. 

We start with a given image / G and consider three different versions 

of the spectral transform. The first one is the scale space from a gradient flow 
approach resembling the TV flow, the last one is obtained from the inverse scale 
space method. In between these methods there is the classical variational regu¬ 
larization. A potentially confusing issue is the fact that time in the inverse scale 
space method rather corresponds to an inverse of the time variable in the other 
models. For this sake we will make change of time variables in the end to ease 
comparison. Moreover, in addition to the spectral representation cj) we introduce 
another one called '0 in the inverse time variable. Note that for small times (j) 
measures changes in high frequencies and 'ip measures changes in low frequen¬ 
cies. In analogy to classical signal processing we call (p wavelength representation 
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and tp frequency representation, respectively. Noticing that a change of variables 
s = i yields 

pCO pOO -| -| -| 

( 8 ) 


r°° 1 11 

4>{t)wit) dt = / ds, 

Jo s s 


Jo Jo 

which motivates the consistency condition 

, 1 . 1 


ijj{s) = respectively (pit) = 


(9) 


' s ' ' ' ' 't t"^' 

that ensures the desirable inverse time relation between frequency and wave¬ 
length representation, (p{t)w{t) dt = tp{s)wi^) ds. 

For simplicity we assume J{u) > 0 for u G X \ { 0 }, which is usually achieved 
by choosing X restricted in the right way (note that the null-space of a con¬ 
vex one-homogeneous functional is a linear subspace of X, 0). E.g. in the case 
of total variation regularization we would consider the subspace of functions 
with vanishing mean value. The general case can be reconstructed by adding 
appropriate nullspace components. The detailed definitions of the spectral rep¬ 
resentations are given as follows: 

Gradient Flo-w Representation: Let uopit) be the solution of 

dtU = -pGF, PGFGdJ{u) ( 10 ) 

for t > 0 with initial value m ( 0 ) = /. Then the corresponding wavelength 

spectral transform is defined by 

(pGFit) = tduUGFit) = -tdtPGFif). ( 11 ) 

We obtain the frequency representation as 

JjGFis) = -^dttUGFi-) = - ^dtPGFi-) = -dsQGFis), ( 12 ) 

S'^ S S’^ s s 

where QGFis) =PGFi^)- 

Variational Representation: Let uvAiit) be the minimizer of 

f\\l + tJiu) ^ min, i.e. uvAiit) = f - tpvM{t), Pvm G dJiuvM)- 

Z u^A. 

( 13 ) 

Then the corresponding high frequency spectral representation is defined by 
(pVAiit) = tduUvMit) = -dtit^dtPvMit)). ( 14 ) 

The low frequency representation can be derived from the equivalent form 

of the variational problem 

s , 


\\v- fWl + Jiv) 


mm, 


(15) 


with the relation wvm(s) = uvAii^), which yields 


’tpVMis) = -dttUvM{-) = sdsis^dgVvMis)). 
s s 


(16) 
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Inverse Scale Space Representation: Let vjsis) be the solution of 

dsqis = f -V, qis € dJ{v) (17) 

for s > 0 with initial value u(0) = 0. Then the corresponding low frequency 
spectral representation is defined by 

'0/s(s) = dsVis{s) = -dssqisis)- (18) 

With uis{t) = visij) obtain 

<(>/s(s) = -t^dtuis{t). (19) 

Note that due to low regularity of J we expect (j)<, to be a measure in time 
(here * stands for either GF, VM, or IS). This is seen immediately from the 
canonical example of / being a (nonlinear) eigenfunction of J: 

Theorem 1. Let f £ X satisfy m for some A > 0 and u = f. Then, with the 
definitions made in (nni) - (Hii) we have 

(l)^{t) = f6^{t), ip*is) = fSx{s) for * = GF,VM,IS. (20) 

Proof. It is straightforward to check that ugf = uvm = (1 “ ^t)+f and the 
a calculation of the second derivative in a distributional sense yields the corre¬ 
sponding (j). In a similar way one can check in the case of the inverse scale space 
method that uis{s) = 0 for s < A and uis(s) = f for s > A is the solution (with 
piecewise linear p), see e.g. [5]. 

The above result confirms our intuition about the spectral decompositions, 
indeed the eigenfunctions give a pure spectrum and the position of certain wave¬ 
lengths respectively frequencies in the spectral domain is proportional respec¬ 
tively inversely proportional to the eigenvalue. 

As in ([S]) we can reconstruct the signal from the spectral response, noticing 
again that we have no nullspace components: 

Theorem 2. Let u* be such that u*(0) = f and uXi) —>■ 0 sufficiently fast for 
t^ oo, for * =GF, VM,IS. Then 

nOO pOO 

f= 4>*{t) dt= ds for * = GF,VM,IS. (21) 

Jo Jo 

Proof. Due to the consistency relation it suffices to verify the reconstruction 
formula only for (j) oi ip. For the appropriate choice the result follows simply by 
integration by parts, noticing u,f{t = 0) = f and all terms at infinity vanish due 
to the decay of u. 

For the sake of brevity we shall restrict ourselves to the gradient flow case in 
the following arguments. An interesting property concerns the orthogonality of 
remaining signal and the spectral transform. In a classical Fourier series we have 
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a decomposition into orthogonal components, so there is natural orthogonality 
between the spectral part at a certain frequency and the remaining signal at lower 
or higher frequency (the sum of orthogonal components with higher or lower 
indices). An analogous property holds for our spectral decomposition, i.e. we 
expect u{t) to be orthogonal to This can be seen from a formal computation 
of in two ways. First of all we have 

^{J{u{t))) = {p{t),dtu{t)) = -|b(t)f. 

On the other, since for one-homogeneous convex functionals J{u{t)) = {p{t), u{t)) 
holds, we find 

j^{J{u[t))) = = {dtp{t),uit)) + lj){t),dtu(t)) = -i((/)(t),u(<))-|b(t)|p 

Hence, comparing the terms we obtain the orthogonality relation. 

Finally let us discuss the spectral response S, which was defined before as 
the L^-norm of (j) in the TV case as mentioned above. This choice is somewhat 
arbitrary and in particular difficult to generalize to other functionals, so we need 
to derive a different version of the spectral response that can be expressed solely 
in terms of the functional J, the L^-norm of /, and the method used to derive 
(j). Let us start with the gradient flow, for which it is natural to investigate the 
energy dissipation, i.e., we compute time derivatives of J. By the chain rule we 
(formally) find 

^J(u(t)) = {p{t),dtu{t)) = 

Moreover, in the case of J differentiable, i.e. p{t) = J'{u{t)), we have 

^J{u{t)) = -2{dtp{t),p{t)) = -2{J''{u{t))dtu{t),p{t)) = 2{J"{u{t))p{t),p{t)), 

which is a nonnegative quantity, since J is convex. For the nonsmooth case we 
might encounter even more interesting spectral responses this way, the nonexis¬ 
tence of a classical second derivative e.g. allows to have concentrated parts. This 
can be made precise again in the case of / being an eigenfunction for the eigen¬ 
value A, i.e., m holds for u = f. Then we know that p{t) = A/ for t < j and 
p{t) = 0 for larger times. Computing |b(t)|| and its derivative we immediately 
find 

( 22 ) 

Since apparently ||/|| is a suitable value for the magnitude, we define the spectral 
response as 

S{t) =t)J^J{u{t)) = \/{(j){t),2tp{t)). (23) 

With this definition we have the following analogue of the Parseval identity, 
which follows again using integration by parts and sufficient decay of u and its 
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derivatives: 

/'OC J poo poo 

Wir = -| = 2 /^ {p{t),u{t)) dt = 2 j^ J{u{t)) dt 

S{tf dt, (24) 

which confirms that the spectral representation encodes the full norm of /. 




Melon image ll</>(t)ILi 


Fig. 2. Spectral resonse for the total variation flow in an image of a melon (left). The 
original definition from m in the middle and the definition (1231) on the right. 

In computational experiments, the behaviour S as defined by (USD appears 
to be more suitable for the spectral decomposition, in particular there are the 
important maxima marking essential changes in the spectrum but less further 
oscillations. This is illustrated for total variation flow on an image already con¬ 
sidered in [15] in Figure [D We finally mentioned that a generalization of this 
definition to variational methods and in particular inverse scale space methods 
is a nontrivial task and beyond the scope of this paper. 

3 Examples 

In the following we discuss three interesting examples of one-homogeneous func¬ 
tionals beyond total variation, which all provide interesting spectral definitions 
in a different context. 

3.1 Recovering Fourier Analysis 

One particular example for a one-homogeneous regularization functional J which 
restores a very classical method is J{u) = ||Fm||i for an orthonormal linear 
transform V, e.g. for V corresponding to frequency analyzing transforms like 
the Fourier or cosine transforms. A similar computation has previously been 
done by Xu and Osher in [25| in which they analyzed Bregman iteration as well 
as the inverse scale space flow for V corresponding to a wavelet transformation. 

For the sake of simplicity let us consider the discrete problem. For any or¬ 
thonormal V we obtain 

min i||u-/ll^+ <||Fu||i, = min ^-/II 2 + 
u Z z Z 

= min h\z - VfWl + t\\z\\i, 

Z Z 
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where we have used the orthonormality of V along with the fact that the 
norm is invariant with respect to multiplication with orthonormal matrices. Note 
that Vf becomes the frequency transform of the input data to which we apply 
regularization. The above minimization problem in z admits a closed form 
solution known as soft-shrinkage, i.e. 


zvMit) = aTgmm-\\z-Vf\\l+t\\z\\i=sign{Vf) max(|f^/| - t, 0), (25) 

2 2 


and hence uvM{t) = V'^zvAiit). The first time derivative of zvM{t) is 


(26) 


Interestingly, we can see that this means that dtZvMit) is in its own negative 
subdifferential, i.e. 

dtZvMit) e —d\\zvM\\i: (27) 

from which we can conclude that zvAiit) = ZQF^t) and uvM{t) = UGpit). The 
second time derivative of z^,{t) (for * G {VM,GF}) becomes 


dtt = sign((t//),) (5t=|(v/)q. 

The wavelength transform function = idttz* is therefore given as 


= (m- 

As we can see the spectral transform <j)l simply reduces to the spectrum of the 
frequency coefficients Vf in this case. Moreover, the first possible definition of 
the spectrum, 

reduces to the sum over the absolute values of the frequencies that have mag¬ 
nitude t, when considered with respect to the z variable. With respect to the 
variable u we obtain 5'“(t) = t\\V'^e\(yf)i\=t\\i, where denotes the vec¬ 

tor with the fth entry being one if |(I^/)z| = t and zero else. Since for random 
DCT coefficients of / the vector e|(yj).|=t is at most one-sparse with probability 
one, the peaks of the spectrum have magnitude |(I^/)i|||ui||i where Vi is the fth 
row of V. Since ||ui||i is not the same for all i and also does not have a specific 
interpretation, definition (l23l) . i.e. 


Sit) = t)J ^J{u{t)) = t)J ^\\Vu{t)\\i = t)J ^IkWIli (28) 

is preferable since it does not depend on the u or z representation. Since the first 
time derivative of ||z(t)||i is iust the number of nonzero entries of z, we obtain 

Sit) = vi{*i i(mi = oi t. 

Finally, let us consider the third possible definition of a spectral definition, 
i.e. the inverse scale space flow. By following we obtain 


r 0 ifsKmi <1, 

1-sign((I//)i) else. 


zisis) = 


(29) 
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such that for t = 1/s we obtain zisit) = zvMit) = ZGF(t) and all three defini¬ 
tions of wavelength and frequency representations coincide. 

Figure 13] shows an example of the above setup using the discrete cosine trans¬ 
form (DCT) for V. As we can see, the low pass reconstructions are obtained as 
hard thresholdings of the coefficients, which is what we refer to as an ideal low 
pass filter. 



DCT coefficient of noisy signal 


Reconstruction z(t) at t = 0.3 




frequency 

Specfrum S(t) of noisy signal 


frequency 
Reconstruction u(t) at t = 0.3 



Fig. 3. Recovering classical spectral analysis: The upper left plot shows a clean signal 
suitable for classical spectral analysis, and below is its noisy version. The upper middle 
image shows the classical processing/denoising of such data. One determines the DCT 
coefficients and eliminates contributions of frequencies whose magnitude is below a 
certain threshold. The lower middle image shows the spectrum S{t) we obtain with the 
proposed approach with J{u) = ||Fu||i for all three variants of the spectral analysis. 
Due to the obvious separation of four of the peaks from the remaining ones, one could 
similarly choose to reconstruct the solution at time t = 0.3, which is equivalent to the 
threshold of 0.3 in the classical sense. The right plots show 2(0.3) and u(0.3), i.e. the 
reconstruction of the ideal filter in the DCT and spatial domains. 


3.2 Total Generalized Variation 

Let us now look at a highly nonlinear variational approach and consider the 
second order total generalized variation (TGV) of [5], which - for sufficiently 
regular u - can be written as J{u) = minvu =«+2 /3||u||i + (l-/3)||Vz||i. 

Figure 131 shows an analysis similar to the classical one of Figure |31 but now 
for the inverse TGV flow using /3 = 0.05. We provide the full movie showing the 
u(t) throughout an evolution of 1500 time steps in the supplementary material 
of this paper. As we can see, applying an ideal low pass filter leads to almost 
perfect reconstructions for signals consisting of piecewise linear parts and jumps. 

This general behavior can be understood by considering eigenfunctions of the 
TGV. Examining our previous observations on the favorable properties of the 
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spectral decomposition techniques on eigenfuctions as well as the observations 
in [5], eigenfunctions should reveal the types of signals the regularization ’likes’ 
in the sense that they can be reconstructed easily. Recalling the TGV eigen¬ 
function analysis in [TWt] one can observe that indeed the appearance of the 
eigenfunction coincides with our numerical observations in the spectral analysis: 
The original signal / is approximated by sequences of piecewise linear functions 
with discontinuities. 


Clean signal 



Low pass filtered signal 



Spectral response 




Low pass filter bands 
- - “Band pass filter bands 
High pass filter bands 



10' 10* 

Time 

High pass filtered signal 



50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500 


Fig. 4. Inverse scale space spectral analysis of a signal using TGV regularization. After 
we added noise to the clean signal (upper left) we obtain the noisy version shown in 
the upper middle. The spectrum we obtain using TGV analysis with S{t) = ||<(>(t)||i is 
shown in the upper right in a semi logarithmic plot. By low pass, band pass or high 
pass Hltering the spectral decomposition (marked blue, red and green in the spectrum) 
we obtain the lower left, middle, and right signals respectively. As we can see one 
can obtain nice reconstructions of the true signal by the low pass filtering. Certain 
frequencies (like the two peaks of the true signal) are contained in intermediate bands 
and can be isolated by band pass filtering. The high pass filter contains mostly noise. 


3.3 Coupled Signal Analysis 

As a third example, we’d like to mention that spectral decomposition could 
also serve as a tool for collaboratively analyzing input data. Just to point out 
possible applications. Figure [S] shows two examples: The left part corresponds 
to an analysis of 15 different input signals / € with the collaborative 

sparse regularization J(u) = ||u||oo,i = Si Similar to the DCT case, 

the problem decouples in one direction and yields £°° regularized subproblems 
in each component. As we can see in the left part of figure [SJ an ideal low-pass 
reconstruction yields exactly those peaks for which all input signals in / had 
non-zero entries. 
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Similar concepts could also be derived for collaborative jumps as shown in the 
right ofO in which we used J{u) = ||Vu||oo,i- As we can see, a strong low-pass 
hlter leads to the best single joint jump approximation in our example. 



Fig. 5. Collaborative spectral analysis. Left: Fifteen different sparse input signals that 
have exactly ten peaks at common positions. Middle left: Ideal low pass filter in the 
||u||oo,i sense - as we can see the location of the common nonzero peaks is recovered. 
Middle right: Fifteen input signals that all have a jump around the position 0.5. Right: 
Ideal low pass filter in the ||Vu||oo,i sense - all fifteen signals are approximated by one 
function which can be interpreted as the optimal single jump approximation of the 
input signals and hence naturally reflects the coarsest scale. 


4 Conclusion 

In this paper the theory of nonlinear spectral representation was generalized and 
extended in several ways. The framework of m was extended from the total 
variation functional to general one-homogeneous convex functionals. Analogue 
transform representations were formulated based on variational regularization 
and on inverse-scale-space m (in addition to the original gradient flow formu¬ 
lation). Moreover, an orthogonality of the decomposition is established and a 
new spectrum is suggested. Roughly, the spectrum measures the degree of activ¬ 
ity of each scale. Here the spectrum is intrinsic to the functional space and can be 
viewed as a natural nonlinear extension of Parseval’s identity. Three examples 
of smoothing one-homogeneous functionals, other than TV, illustrate possible 
benefits of this approach for enhanced image and signal representation. 
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